Robotic device with time-of-flight proximity sensing system

ABSTRACT

A robotic device including one or more proximity sensing systems coupled to various portions of a robot body. The proximity sensing systems detect a distance of an object about the robot body and the robotic device reacts based on the detected distance. The proximity sensing systems obtain a three-dimensional (3D) profile of the object to determine a category of the object. The distance of the object is detected multiple times in a sequence to determine a movement path of the object.

BACKGROUND Technical Field

The present disclosure is directed to robotic devices includingproximity sensors for detecting an approaching object.

Description of the Related Art

Traditional robots are widely used in industrial environments, includingheavy material handling, welding, and painting in the automotiveindustry. Usually, the traditional robots are kept apart from humanbeings for safety reasons. Along with the ongoing Industry 4.0, use ofrobots is being extended into many different industry segments andresidential environments. In many emerging robotic usage scenarios,robots are required to work with human beings side by side in anon-controlled environment. For example, robots are moving freely andare not locked away or fixed in a working area. As such, variousexpected or unexpected objects could appear about a robotic device. Itis important to design a robot which is able to be aware of thesurrounding objects and react correspondingly.

Traditional object sensing techniques are proven to be deficient forsuch new robot use scenarios. Ultrasonic sensors must view a surfacesquarely or perpendicularly to receive ample sound echo. Also, reliablesensing by an ultrasonic sensor requires a minimum target surface area,which is specified for each sensor type. While ultrasonic sensorsexhibit good immunity to background noise, these ultrasonic sensors arestill likely to falsely respond to some loud noises, like the “hissing”sound produced by air hoses and relief valves.

Proximity style ultrasonic sensors require time for a transducer to stopringing after each transmission burst, before they are ready to receivereturned echoes. As a result, sensor response times are typicallyslower, which is disadvantageous for a robotic device responding in anun-controlled working environment. Ultrasonic sensors require a minimumsensing distance; namely, there are blind spots for ultrasonic sensors.Changes in the environment, such as temperature, pressure, humidity, airturbulence, and airborne particles, affect ultrasonic response. Targetsof low density, like foam and cloth, tend to absorb sound energy, andmay be difficult to sense at long range to an ultrasonic sensor.

BRIEF SUMMARY

Examples of the present disclosure are directed to a robotic deviceincluding one or more proximity sensing systems coupled to variousportions of a robot body of the robotic device. The proximity sensingsystems detect a distance of an object about the robot body such thatreactions may be made based on the detected distance of the object. Theproximity sensing systems may be controlled to obtain athree-dimensional (3D) profile of the object. Such detected 3D profilesmay be compared with stored 3D profiles of known object categories suchthat the robotic device is able to categorize the object and reactaccordingly. In obtaining the 3D profile of the object, the detectionrange of the proximity sensing system may be adjusted.

To maintain a balance between processing speed and accuracy, thedetection resolution, namely the amount of distance information detectedon an object, may be adjusted based on the distance of the object. Whenan object is closer to the robot body, detection resolution will beincreased. The detection resolution may be in a space domain, namely anumber of distance detections for a unit area of the object, or in atime domain, namely a number of distance detections on an object in aunit period of time.

With multiple detected distance information of an object in a timesequence, a movement direction and a movement speed of the object isdetermined. Such movement direction and movement speed of the object maybe used to predict a further movement of the object and may be comparedwith a movement trajectory of a moveable portion of the robot body todetermine a reaction of the robotic device to the object.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

In the drawings, identical reference numbers identify similar elementsor acts unless the context indicates otherwise. The sizes and relativepositions of elements in the drawings are not necessarily drawn toscale.

FIG. 1 illustrates an example robotic system;

FIG. 2 illustrates an example proximity sensing system with a multi-zonesensing scheme;

FIG. 3 illustrates an example operation of the example proximity sensingsystem of FIG. 2;

FIG. 4 illustrates another example operation of the example proximitysensing system of FIG. 2;

FIG. 5 illustrates another example operation of the example proximitysensing system of FIG. 2;

FIG. 6 illustrates example reaction zones of a robotic system;

FIG. 7 illustrates an example operation process of operating a roboticsystem;

FIG. 8 illustrates an example time-of-flight proximity sensor;

FIG. 9 illustrates a functional block diagram of an exampletime-of-flight proximity sensor;

FIG. 10 illustrates a functional diagram of an example single zonetime-of-flight proximity sensor;

FIG. 11 illustrates a functional diagram of a multi-zone time-of-flightproximity sensor;

FIGS. 12 and 13 are example graphs illustrating operations of an exampletime-of-flight proximity sensor;

FIG. 14 illustrates an example histogram generated by an exampletime-of-flight proximity sensor; and

FIG. 15 illustrates a diagram illustrating an operation environment ofan example multi-zone time-of-flight proximity sensor.

DETAILED DESCRIPTION

In the following description, certain specific details are set forth inorder to provide a thorough understanding of various examples of thedisclosure. However, one skilled in the art will understand that thedisclosure may be practiced without these specific details. In otherinstances, well-known structures associated with electronic componentsand fabrication techniques have not been described in detail to avoidunnecessarily obscuring the descriptions of the examples of the presentdisclosure. The drawings are not necessarily drawn to scale, and somefeatures are enlarged to provide a more clear view of particularfeatures.

Unless the context requires otherwise, throughout the specification andclaims that follow, the word “comprise” and variations thereof, such as“comprises” and “comprising,” are to be construed in an open, inclusivesense, that is, as “including, but not limited to.”

The use of ordinals such as first, second and third does not necessarilyimply a ranked sense of order, but rather may only distinguish betweenmultiple instances of an act or structure.

Reference throughout this specification to “one example” or “an example”means that a particular feature, structure or characteristic describedin connection with the example is included in at least one example.Thus, the appearances of the phrases “in one example” or “in an example”in various places throughout this specification are not necessarily allreferring to the same example. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner inone or more examples.

As used in this specification and the appended claims, the singularforms “a,” “an,” and “the” include plural referents unless the contentclearly dictates otherwise. It should also be noted that the term “or”is generally employed in its sense including “and/or” unless the contentclearly dictates otherwise.

The present disclosure is generally directed to a robotic system havingmultiple proximity sensing systems coupled to multiple portions of arobot body of the robotic system. The multiple proximity sensing systemseach may detect a distance of an object, e.g., a human being or anobstacle, which the robot body may need to avoid, or to work togetherwith. Based on the detected distance, the robotic system may react tothe object being detected. The reactions may be in the operation of theproximity sensing system or may be in the operation of the robot body.With the included time-of-flight proximity sensors, the proximitysensing systems each may achieve much higher sensing resolution andspeed that enable more comprehensive processing of the sensed data asincluded in the current disclosure. An illustrative example roboticsystem 100 is shown in FIG. 1. In this example, robotic system 100 mayinclude a robot body 110, one or more proximity sensing systems 120,shown as 120 a, 120 b, 120 c, and a control system 150. Control system150 may be coupled to robot body 110 and proximity sensing system 120electrically or communicatively to receive data therefrom and to controlthe operations and functions of robot body 110 and proximity sensingsystem 120. FIG. 1 only shows, for simplicity, the signal coupling linksbetween control system 150 and proximity sensing system 120, which doesnot limit the scope of the disclosure.

Robot body 110 may include a base 112, and one or more moveable portions114 (e.g., jointed arms), that are coupled to base 112, shown as pivotedthrough joints 116 for illustrative purposes only. One or more moveableportion 114 may include an end effector 118 that is coupled thereto. Thecoupling of moveable portion 114 and end effector 118 may be achievedthrough one or more joints 116. It should be appreciated that in thedescription herein, the base 112 and moveable portion 114 are definedrelative to one another only for descriptive purposes, and neither limitthe scope of the disclosure. For example, the base 112 might be amoveable portion relative to its own “base.” Moveable portion 114 b maybe treated as a base for moveable portion 114 in the sense that moveableportion 114 a moves, e.g., rotates, relative to moveable portion 114 b,and portion 114 b does not move relative to moveable portion 114 a.Moveable portion 114 b is moveable relative to base 112. It should befurther appreciated that robot body 110 as shown in FIG. 1 is just anillustrative example of a robot body and does not limit the scope of thedisclosure. A robot body may include different structures or additionalstructures than the example robot body 110 of FIG. 1, which are allincluded in the disclosure. For example, example robot body 110 may be aportion of a larger scale robot that overall serves as base 112 to robotbody 110. Base 112 may be any platform of a robot or may be positionedon any platform of a robot, including, but not limited to, a locomotionplatform capable of travelling at least one of in the air, in the wateror on the ground.

Each proximity sensing system 120 may include multiple proximity sensingsensors 120, and may be coupled to various portions of robot body 110 invarious means, which are all included in the disclosure. In an example,for a proximity sensing system 120 attached to a portion of a robotbody, e.g., proximity sensing system 120 b attached to moveable portion114 b, the multiple sensors thereof may be arranged to have a detectionrange that covers the full relative movement and reachable region of therelative moveable portion(s), e.g., moveable portion 114 a, that ispivoted directly or indirectly thereto. For example, as both moveableportion 114 a (that includes end portion 118) and moveable portion 114 bare moveable relative to base 112 and are pivoted to base 112, aproximity sensing system 120 attached to base 112 may be arranged tohave a detection range that covers the full relative movement andreachable region of moveable portions 114 a and 114 b. A relativemovement region is a region of movement of a moveable portion 114relative to its base 112, which is the immediately adjacent piecerelative to the moveable portion. For example, the relative movementregion of moveable portion 114 a is relative to moveable portion 114 b,which is a base for moveable portion 114 a.

The detection range of a proximity sensing system 120 may be achievedthrough the fixed positioning arrangement of the one or more proximitysensors thereof and may be achieved through arrangement of the movementranges, namely scanning ranges, of the proximity sensors 120 thereof.

Control system 150 may include a proximity measurement receiver 152, asensor controller 154, a robot controller 156, and a coordinationcontroller 158. Proximity measurement receiver 152 may be electricallyor communicatively coupled to each proximity sensing system 120, or eachproximity sensors 120 thereof, to receive the sensing data obtained bythe proximity sensing systems 120.

Sensor controller 154 may be at least one of electrically orcommunicatively coupled to each proximity sensing systems 120 or eachproximity sensor 120 thereof, to control the operations of proximitysensing systems 120 or the proximity sensors 120 thereof. Thecontrolling of the sensor operations may include adjusting at least oneof a detection range, a detection resolution, or a scanning speed of aproximity sensing system 120 or a proximity sensor thereof. A detectionrange is a range of a distance of an object(s) which a proximity sensor120 is capable to detect. A detection range of a proximity sensingsystem 120 is the combination of all the detection ranges of theproximity sensors 120 included in the proximity sensing system 120. Adetection resolution may include a resolution in space domain (“spatialresolution”) or a resolution in time domain (“detection frequency”).Detection frequency is a frequency of a light signal emitting aproximity sensor. A scanning speed is an area detected/scanned by aproximity sensor 120 within a unit of time. Both detection resolutionand scanning speed of a proximity sensor 120 are related to an amount ofdistance information the proximity sensor 120 obtained from an objectand the processing of the information.

The controlling of at least one of proximity sensing system 120 or theproximity sensors 120 thereof may be effected at least one ofelectrically or mechanically. For example, the electrical control may beachieved through a gate control signal controlling the light emittingand receiving of a proximity sensor 120. The mechanical control may beachieved through controlling a mini-motor(s) coupled to a proximitysensor(s) of a proximity sensing system(s) 120.

FIG. 2 shows an example operation environment 200 to receive data from,and to control, a proximity sensing system 120. As shown in FIG. 2, anexample proximity sensing system 120 may include multiple, shown asfour, proximity sensors 120(1), 120(2), 120(3) and 120(4), which may bedistinct proximity sensors or zones of a single proximity sensor asdescribed herein. Proximity sensors 120(1), 120(2), 120(3) and 120(4)each may detect a respective region of an object 210, referred to asobject zones 210(1), 210(2), 210(3) and 210(4). Specifically, proximitysensor 120(1) detects a distance of object zone 210(1) of object 210 toproximity sensor 120(1); proximity sensor 120(2) detects a distance ofobject zone 210(2) to proximity sensor 120(2); proximity sensor 120(3)detects a distance of object zone 210(3) to proximity sensor 120(3); andproximity sensor 120(4) detects a distance of object zone 210(4) toproximity sensor 120(4). It should be appreciated that each proximitysensor 120 may be able to detect multi-zone information of thecorresponding object zone 210, which is described further herein.

Each proximity sensor 120 may be individually coupled to one or moremini-motors (not shown for simplicity) such that sensor controller 154can control at least one of the movement or position of each proximitysensor 120. As shown in FIG. 2 as an illustrative example, movement 220of proximity sensors 120, and the related detection range, may includelinear movements in any directions within the X-Y-Z three-dimensionalsphere. Movement 220 of a proximity sensor 120 may also include tiltingor rotation in various directions. As appreciated, a movement 220 ofproximity sensor 120 may change its detection range. The ability ofsensor controller 154 to change the detection range of each individualproximity sensor 120 brings about various technical advantages. Forexample, proximity sensing systems 120 may individually or incoordination detect a three-dimensional (“3D”) profile of an object 210.For example, by comparing the distance information/readings detected bythe multiple proximity sensors 120 of a proximity sensing system 120,the relative depth of the multiple zones 210 could be determined, whichis part of the 3D profile of object 210.

By moving proximity sensors 120 in various ways, more 3D profileinformation of object 210 may be obtained. For example, when thedistances of object zone 210(1) and object zone 210(2) detected byadjacent proximity sensors 120(1) and 120(2) are within a firstthreshold range of one another, which, e.g., indicates sufficientflatness or smooth transition between object zones 210(1) and 210(2),sensor controller 154 may move the detection range of at least one ofproximity sensors 120(1), 120(2) away from one another so that the edgeof the flatness or the smooth transition area may be detected. When thedistances of object zone 210(1) and object zone 210(2) detected byadjacent proximity sensors 120(1) and 120(2) are beyond a secondthreshold range of one another, which, e.g., indicates a sharp depthchange between object zones 210(1) and 210(2), sensor controller 154 maycontrol to move the detection range of at least one of proximity sensors120(1), 120(20 toward one another so that more depth information betweenthe original object zones 210(1) and 210(2) may be obtained. Such 3Dprofile information may help to single out an object 210 from at leastone of the background or an adjacent different object 210. Moredescription about multi-object detection is described herein.

FIGS. 3-5 show example operations of adjusting the detection ranges ofindividual proximity sensors 120 in detecting a 3D profile of anobject(s). FIG. 3 shows that proximity sensors 120(1), 120(2), 120(3),120(4) detect object zones 210(1), 210(2), 210(3), 210(4), which alloverlap a vertical pole 340 in the example residential environment. Thedistance values 322(1), 322(2), 322(3), 322(4) obtained by proximitysensors 120(1), 120(2), 120(3), 120(4) are all example value “5”, whichindicates sufficient flatness between/among object zones 210(1), 210(2),210(3), 210(4). This “flatness” indicates the proximity sensors are allrelatively equally spaced from the object being detected.

In FIG. 4, the sensor controller 154 controls proximity sensors 120(2)and 120(4) to move their detection range away from those of proximitysensors 120(1) and 120(3), and now the respective zones 210(2) and210(4) cover a further background item. This is a change in the field ofview of the proximity sensors 120(2) and 120(4). The distance values322(1) and 322(3) obtained by proximity sensors 120(1), 120(3), remain“5”, while distance values 322(2) and 322(4) obtained by proximitysensors 120(2), 120(4) are now “10”, which shows a sharp variation indepth between zones 210(1)/210(3) and 210(2)/210(4) and identifies anedge of the vertical pole 340 and a different object spaced further fromthe vertical pole 340.

FIG. 5 shows that sensor controller 154 moves proximity sensors 120(1),120(2), 120(3), 120(4) together further to the right from the objectzones 210(2) and 210(4) of FIG. 4. Now in FIG. 5, object zones 210(1),210(3) of proximity sensors 120(1) and 120(3) cover the background itemand zones 210(2), 210(4) of proximity sensors 120(2) and 120(4) cover anew object (the two legs of the human being 540). The distance values322(1), 322(2), 322(3), 322(4) obtained by proximity sensors 120(1),120(2), 120(3), 120(4) are now “10”, “6”, “10”, “6.5”, which indicatesthat object zones 210(2) and 210(4) are of a different object thanobject zones 210(1) and 210(3). The difference between zones 210(2) and210(4), “0.5,” may meet a second threshold and may require thatproximity of some sensors 120(1), 120(2), 120(3), 120(4) be moved closerto one another to obtain more depth information between object zones210(2) and 210(4) to further clarify the 3D profile information aboutthe object(s). The control system 150 may include various thresholdsregarding the proximity sensor readings to control and adjust the fieldsof view of the proximity sensors 120, which are all included in thedisclosure. Each of the proximity sensors may have a micro-motorassociated with the proximity sensor to enable small adjustments of theposition of the proximity sensor to change a field of view of theproximity sensor. With the variations in movement of the proximitysensors, the system can has overlapping fields of view of the proximitysensors and can compile the various depth measurements with the variouspositions to create the 3D profile of the object.

Such capacity to determine a 3D profile of an object 210 is advantageousfor robot system 100 in that robot system 100 may choose to takedifferent actions with respect to the detected objects 210 of different3D profiles. For example, robot system 100 may differentiate between ahuman being and a background obstacle. Robot system 100 may alsodifferentiate between an intended object 210 to work with and a foreignobject 210 protruding into a working space of robot body 110.

In an example, coordination controller 158 may be configured to comparethe detected 3D profile of an object 210 with stored, e.g., on a memoryassociated with control system 150 (not shown for simplicity), 3Dprofiles of various types of objects to determine acategory/classification of the detected object 210. For example, a 3Dprofile of a human head may be used to determine whether a detectedobject 210 is a human-being. The stored 3D profiles may not need to be afull profile of the types of object and may include the 3Dcharacteristics and identifiers of the types of objects. For example,sensor controller 154 may adjust a detection resolution of a proximitysensing system 120 or the proximity sensors thereof based on theclassification of the detected object. For example, more detectionresolution may be provided for an object 210 determined as a human beingbased on the 3D profiles.

Further, sensor controller 154 may control the operation of one or moreproximity sensing system 120 based on a detected distance of an object210. Sensor controller 154 may provide control signals to adjust atleast one of a detection resolution or a scanning speed of a proximitysensing system 120 or the proximity sensors thereof. The detectionresolution may include a spatial resolution and detection frequency. Aspatial resolution refers to a number of detected object zones, e.g.,210(1), 210(2), 210(3), 210(4), within a unit area of an object 210. Forexample, the spatial resolution may be adjusted through adjusting thefields of view of proximity sensors 120 to be closer to one another orbe turned away from one another. A detection frequency refers to anumber of detection readings performed by proximity sensor 120 within aunit time period. Related to the detection resolutions, a scanning speedof moving a proximity sensing system 120 to scan a larger area may alsobe controlled by sensor controller 154. For example, sensor controller154 may at least one of increase the detection resolution or decreasethe scanning speed of the multiple proximity sensors 120 of proximitysensing system 120 when a detected distance of object 210 is smallerthan a third threshold, e.g., within critical zone 620 of FIG. 6 Sensorcontroller 154 may at least one of decrease the detection resolution orincrease the scanning speed of the multiple proximity sensors 120 ofproximity sensing system 120, when the detected distance of the object210 is larger than a fourth threshold.

The adjustment of a detection resolution in a space domain may beachieved through various approaches. One approach is to physically movethe detection ranges of multiple proximity sensors 120 to be closer orfurther apart. Another approach is to selectively process or not toprocess the sensing data obtained by one or more proximity sensors 120.Other approaches to adjust a detection resolution in space domain arealso possible, and are included in the disclosure.

It should be appreciated that when a time-of-flight proximity sensor isused for proximity sensors 120, each time-of-flight proximity sensor isable to detect a 3D profile of the relevant zone 210 in the detectionrange of the time-of-flight sensor. The detections zones of the singletime-of-flight proximity sensor may also be controlled by sensorcontroller 154, e.g., by controlling the light emission angles of thelight emitting element. Further details about multi-zone time-of-flightproximity sensor are provided herein.

Referring back to FIG. 1, robot controller 156 may be configured tocontrol an operation of robot body 110 based on a detected distance ofan object 210. As described herein, the one or more proximity sensingsystems 120 or the proximity sensors thereof may each detect variousdistance data of an object at different object zones 210, and a 3Dprofile of the object 210 may be obtained. The control of the operationof robot body 110 may be based on the different distances of the objectzones 210, and may be based on the 3D profile of the object 210.

FIG. 6 shows an example operation environment 600 of robot body 600under control of controller 150 (not shown). Referring now to FIG. 6,robot controller 156 may define multiple reaction zones about robot body110. The multiple reaction zones may be defined relative to a moveableportion 114. That is, for different moveable portions 114, differentreaction zones may be defined. The reaction zones are defined in amanner that, when an object 210 is detected within a specific reactionzone, the corresponding reactions will be taken by robot body 110 undercontrol of robot controller 156. FIG. 6 shows, as an illustrativeexample, a work zone 610, a critical zone 620, and a watch zone 630defined relative to moveable portion 114 a including end effector 118.As an illustrative example, the reaction zones 610, 620, 630 are definedbased on a distance 640 to end effector 118 in different directions.That is, the border of each of zones 610, 620, 630 may have differentdistance to end effector 118 from different directions. For simplicityand illustrative purposes, in FIG. 6, reaction zones 610, 620, 630 areillustrated with two-dimensional eclipse-type shapes, which is notlimiting. The reaction zones may be any two-dimensional orthree-dimensional spheres with various shapes, which are all included inthe disclosure. The fast detection and computation capacity of time-offlight proximity sensors 120 enables refined definition of reactionzones.

In an example, as reaction zones 610, 620, 630 are defined with respectto the corresponding moveable portion 114 a, when a base of thecorresponding moveable portion (here moveable portion 114 b) moves, thereaction zones 610, 620, 630 also move. When the moveable portion 114 bstays fixed, the reaction zones 610, 620, 630 may not change with themovement of the corresponding moveable portion 114 a because the settingup of reaction zones 610, 620, 630 already considered the movement ofthe corresponding moveable portion 114 a.

Robot controller 156 may control operations of robot body 110 based on adetected distance of an object 210 mapped into the reaction zones. Forexample, robot controller 156 may control an operation speed of moveableportion 114 a based on the detected distance of object 210 mapped withinzones 610, 620 or 630. It should be appreciated that, although thereaction zones 610, 620 and 630 are defined relative to moveable portion114 a, it does not mean that robot controller 156 will only control theoperation of moveable portion 114 a in response to an object 210 beingdetected within one of the reaction zones 610, 620, 630. Robotcontroller 156 may also control the operation of other moveable portions114 and base 112 in response to an object 210 being detected within areaction zone 610, 620, 630 defined with respect to moveable portion 114a. For example, for an object 210 detected within watch zone 630 andoutside critical zone 620, robot controller 156 may control robot body110 to remain the normal operation speed for moveable portion 114 a andmay alarm at least one of robot body 110 and the detected object 210 ifa movement of the relative base of moveable portion 114 a, e.g.,moveable portion 114 b and base 112 is moving toward the detected object210.

If an object 210 is detected within critical zone 620 and outside workzone 610, robot controller 156 can adjust a speed of movement of themoveable portion 114 a to avoid a collision with the object. The robotcontroller 156 can, for example, control robot body 110 to reduce theoperation speed of moveable portion 114 a. Alternatively, if themoveable portion is already on a path away from the object, or thetrajectory of the object, the controller may not change the path or mayspeed up the movement of the moveable portion along the existing path ifthat trajectory is away from the possible collision.

If an object 210 is detected within work zone 610, robot controller 156may control robot body 110 to pause the operation of moveable portion114 a.

In an example, robot controller 156 may control a reaction of robot body110 based on a determined movement path of the object 210 as compared toa movement trajectory of moveable portion 114 a in controlling robotbody 110. The movement path of object 210 may include a movementdirection of object 210. For example, as shown in FIG. 6, the movementpath 652 of object A may be estimated as approaching the work zone 610of moveable portion 114 a based on the detected distances/positionsdetected from T0 to T3. The movement path 654 of object B may beestimated as leaving watch zone 630 based on the positions/distancesdetected from T0 to T2. Object C may be detected as being stable (notmoving) within watch zone 630. Further, the determination of a movementpath of object 210 may include determining a movement speed of object210. With the determined movement path of object 210 and the movementtrajectory of moveable portion 114 a, robot controller 156 may furtherrefine the control of robot body 110.

Further, the mapping of the detected distance of object 210 into thereaction zones also helps the control of proximity sensing systems 120by at least one of sensor controller 154 or coordination controller 158as described herein.

With multiple proximity sensing systems 120 are attached to differentportions of robot body 110, the distance readings of different proximitysensing systems 120 may not be fully consistent with one another. Eachof these proximity sensors may have different distances or detectionzones that they monitor. For example, a sensor on moveable portion 114 bmay be a first distance threshold detector, such as a distance that is adistance past a boundary of the watch zone 630. The sensor on themoveable portion 114 b may have a larger range than the sensors onmoveable portion 114 a and may act as a first detection, a warning thatan object may be moving towards the watch zone.

Alternatively or in addition, the sensor on the movable portion 1114 bmay be angled to have a field of view that does not overlap with thefield of view of the sensors on the moveable portion 114 a. Having adifferent field of view, such as toward the floor allows for detectionof object that may not be within the watch zone 630 or closer to the endeffector 118. The controller takes the various data point regarding anenvironment around the robot body to make decisions about movement ofthe various moveable portions.

In an example, robot controller 156 may assign different priorities tothe distance readings from different proximity sensing systems 120 basedon the detected distance of object 210. For example, if the detecteddistance is smaller than a fifth threshold distance, proximity sensingsystem 120 a attached to moveable portion 114 a may have a priority overproximity sensing system 120 b attached to moveable portion 114 b, whichis a base for moveable portion 114 a. When the detected distance ofobject 210 is greater than the fifth threshold distance, proximitysensing system 120 b attached to moveable portion 114 b may have apriority over proximity sensing system 120 a attached to moveableportion 114 a. Such configuration may help to ensure that in an urgentscenario, namely object 210 is very close to end effector 118, the morestraightforward information obtained by sensors closer to the involvedrobot portion, i.e., moveable portion 114 a, is first considered. In aless urgent scenario, readings of proximity sensors attached to arelatively fixed portion of robot body 110, i.e., proximity sensors 120attached to moveable portion 114 b, may be able to obtain morecomprehensive information about an object 210.

Referring back to FIG. 1, coordination controller 158 may be configuredto coordinate the control of robot body 110 and the control of proximitysensing systems 120. Coordination control unit 158 may achieve thecoordination of the controls through communication with at least one ofsensor controller 154 or robot controller 156. Coordination control unit158 may also directly control the operation of at least one of proximitysensing systems 120 or robot body 110. In an example, coordinationcontroller 158 may be configured to compare a detected 3D profile of anobject 210 with a stored parameter 3D profile of known object categoriesto determine a category/classification of the detected object 210.

FIG. 7 shows an example operation process 700. Referring to FIG. 7, inexample operation 710, one or more proximity sensing systems 120 (or oneor more proximity sensors thereof) may detect a distance of an object210. The distance may include multiple distance values detected bymultiple proximity sensors 120 or may include multiple distance valuesdetected by a single proximity sensor 120, e.g., a time-of-flightproximity sensor.

In example operation 720, proximity measurement receiver 152 may receivethe detected distance of object 210 from proximity sensing systems 120and may map the received distance into the reaction zones of variousmoveable portions 114 of robot body 110. As described herein, differentreaction zones may be defined relative to different moveable portions114 of robot body 110. Proximity measurement receiver 152 may map thereceived distance of object 210 into different zones of differentmoveable portions 114 for further processing and the related control ofat least one of proximity sensing systems 120 or robot body 110.

In the description herein, reaction zones of example moveable portion114 a may be used as an illustrative example.

Referring to FIGS. 6-7 together, if the detected distance is mapped intowork zone 610, robot controller 156 may control robot body 110 to stopthe operation, e.g., movement, of moveable portion 114 a, in exampleoperation 730.

If the detected distance of object 210 is mapped into watch zone 630,robot controller 156 may control robot body 110 to maintain the normaloperation of the status quo operation state, e.g., robot body 110 mayalready be operating under a reduced speed state due to other detectedobject 210. At the same while, sensor controller 154 may control one ormore proximity sensing systems 120 to obtain a 3D profile of object 210,in example operation 740. As described herein, the obtaining a 3Dprofile of object 210 may include adjusting the detection ranges of oneor more proximity sensors 120 based on the detected distance of object210.

In example operation 742, coordination controller 158 may compare theobtained 3D profile of object 210 with stored parameter 3D profiles todetermine a category/classification of object 210. For example, based onthe comparison result, coordination controller 158 may be able todetermine whether the object 210 is an object to be avoided, e.g., ahuman being, an obstacle, or an object to be worked with, e.g., to grabby end effector 118. The parameter 3D profiles may be pre-determined andstored in relation to control system 150 and may be continuously anddynamically updated. For different operation scenarios of robot system100, different parameter 3D profiles may be applied.

In example operation 744, robot controller 156 may identify and registerobject 210 (that is detected within watch zone 630) in a watch list forfurther distance detection analysis. For example, robot controller 156may register the object 210 as a potential target for tracking amovement path.

If object 210 is detected within critical zone 620, robot controller 156may control robot body 110 to reduce operation speed of moveable portion114 a and other portions of robot body 110, in example operation 750. Inan example, robot body 110 may include two or more speed profiles asoperation modes. When an object 210 is detected within critical zone620, robot controller 156 may not stop the operation of robot body 110,and may switch or maintain its operation on a lower speed mode.

Further, for an object 210 within critical zone 620, sensor controller154 may control proximity sensing system 120 to increase the proximitydetection/reading resolution on the object 210, in example operation752. The increased detection resolution may include an increased spatialresolution by increasing the number of proximity sensor 120 readings onobject 210. The increased detection resolution may include an increaseddetection frequency on object 210.

Alternatively or additionally, sensor controller 154 may controlproximity sensing system 120 to reduce the scanning speed relevant toobject 210 within critical zone 620 such that more distance readings maybe obtain on object 210, in example operation 754

Further, for an object 210 within critical zone 620, robot controller156 may start tracking and obtaining the movement path includingmovement direction and movement speed of object 210, in exampleoperation 756. The movement path of object 210 may be compared with amovement trajectory of moveable portion 114 a of robot body 110 todetermine further reactions to object 210 being in critical zone 620.

In an example, a proximity sensor 120 is a time-of-flight proximitysensor. Time-of-Flight range imaging measures a depth of a 3D object 210by calculating a time duration that an emitted light takes to travel toand bounce back from the 3D object 210. For example, acontinuously-modulated invisible light wave, e.g., a laser or aninfrared (IR) light beam, is emitted from a light emitting unit, and thephase delay between the received bounce-back light and the originalemitted light is detected and calculated to determine the depth, i.e., adistance between a time-of-flight sensor and the object.

A time-of-flight proximity sensor 120 may include a Single PhotonAvalanche Diode (SPAD), which is a p-n junction device biased beyond itsbreakdown region.

Referring to FIG. 8, an example circuit implementation of a pixel 800(“Z” pixel) of a time-of-flight proximity sensor 120 is shown. As shownin FIG. 8, example circuit implementation 800 includes an example rangepixel circuit 810, an example readout circuit 820 and a readout switch830.

Range pixel circuit 810 includes a fast pinned photodiode (FAST PPD)812, e.g., a pinned single photo avalanche diode (SPAD), coupled to a3-bin demodulation unit 814. Demodulation unit 814 includes threein-parallel transfer-to-memory gates (TGMEM1, TGMEM2, TGMEM3) modulated(through gate control signals) with an out-of-phase angle of 120° amongone another, three diodes (e.g., same type of pinned photodiode as FASTPPD 812) coupled to the gates TGMEM1, TGMEM2, TGMEM3, respectively, andfunctioning as memories to save the electrical potential at node “A”,and three transfer-to-readout gates TGRD1, TGRD2, TGRD3 coupled to thememory photodiodes PPD1, PPD1, PPD3, respectively, to selectively outputthe saved electrical potential information of the three diodes at node“B”.

Readout circuit 820 may include one or more readout element 825. One ormore range pixel circuit 810 may share a same readout circuit 820 or asame readout element 825.

Other demodulation configurations are also possible and included in thedisclosure. For example a 4-bin demodulation unit (with 90° our-of-phaseangles) or a 2-bin demodulation unit (with 180° our-of-phase angles) areall possible configurations.

In operation, demodulation unit 814 is configured to sample the incomingmodulated non-visible light wave, e.g., IR beam, three times permodulation period. Each sampling saves the photo-generated electricalpotential at node “A” for the one third fraction of the modulationperiod under, e.g., a modulation frequency of 20 MHz. The saved sampledreadings will be fed to a data processing unit through readout circuit820 and the modulation signal will be reconstructed through dataprocessing based on the sampled readings. Further elaboration of thedata processing details are not required for the appreciation of thedisclosure and may be omitted for simplicity purposes.

Switch 830 may be switched on/off by control signal “CONTROL” fordistance readings of range pixel circuit 810 to be read or not to beread by readout circuit 820/readout element 825. Such control may beused to achieve the detecting resolution of measuring system 120 on anobject 210. For example, sensor controller 154 may choose not to readand process readings of a range pixel circuit 810 to reduce thedetection resolution and increase data processing speed and efficiency,e.g., for an object 210 detected as far away from robot body 110.

FIG. 9 is an example functional block diagram of a time-of-flight (TOF)proximity sensor 120. In the example of FIG. 9, the TOF proximity sensor120 includes a light source 1300, which is, for example, a laser diodesuch as a vertical-cavity surface-emitting laser (VCSEL) for generatingthe transmitted optical pulse signal designated as 1302 in FIG. 9. Thetransmitted optical pulse signal 1302 is transmitted in the detectionrange of the light source 1300. In the example of FIG. 9, thetransmitted optical pulse signal 1302 is transmitted through aprojection lens 1304 to focus the transmitted optical pulse signals 1302so as to provide the desired detection range. Sensor controller 154 maybe configured to adjust the projection lens 1304 to adjust the detectionrange of TOF proximity sensor 120. The projection lens 1304 is anoptional component, with some examples of the sensor not including theprojection lens.

The reflected or return optical pulse signal is designated as 1306 inFIG. 9 and corresponds to a portion of the transmitted optical pulsesignal 1302 that is reflected off objects within the detection range.One such object 210 is shown in FIG. 9. The return optical pulse signal1306 propagates back to the TOF proximity sensor 120 and is receivedthrough a return lens 1309 which provides another control for thedetection range from the receiving side of TOF proximity sensor 120. Thereturn lens 1309 directs the return optical pulse signal 1306 to rangeestimation circuitry 1310 for generating the imaging distance D_(TOF)and signal amplitude SA for each object 210. The return lens 1309 isalso an optional component and, thus, some examples of the TOF proximitysensor 120 may not include the return lens.

In the example of FIG. 9, the range estimation circuitry 1310 includes areturn single-photon avalanche diode (SPAD) array 1312, which receivesthe returned optical pulse signal 1306 via the lens 1309. The SPAD array1312 may include a large number of SPAD pixels, e.g., range pixel 800 ofFIG. 8, each cell including a SPAD for sensing a photon of the returnoptical pulse signal 1306. In some examples of the TOF proximity sensor120, the lens 1309 directs reflected optical pulse signals 1306 fromseparate spatial zones within the detection range of the sensor 120 tocertain groups of SPAD pixels or zones of SPAD pixels in the return SPADarray 1312, as will be described in more detail below.

Each SPAD pixel in the return SPAD array 1312 provides an output pulseor SPAD event when a photon in the form of the return optical pulsesignal 1306 is detected by that pixel in the return SPAD array. A delaydetection and processing circuit 1314 in the range estimation circuitry1310 determines a delay time between transmission of the transmittedoptical pulse signal 1302 as sensed by a reference SPAD array 1316 and aSPAD event detected by the return SPAD array 1312. The reference SPADarray 1316 is discussed in more detail below. The SPAD event detected bythe return SPAD array 1312 corresponds to receipt of the return opticalpulse signal 306 at the return SPAD array. In this way, by detectingthese SPAD events, the delay detection and processing circuit 1314estimates an arrival time of the return optical pulse signal 306. Thedelay detection and processing circuit 1314 then determines the time offlight (TOF) based upon the difference between the transmission time ofthe transmitted optical pulse signal 1302 as sensed by the referenceSPAD array 1316 and the arrival time of the return optical pulse signal1306 as sensed by the SPAD array 1312. From the determined time offlight (TOF), the delay detection and processing circuit 1314 generatesthe range estimation signal RE indicating the detected distance D_(TOF)between the object 210 and the TOF proximity sensor 120. As appreciated,the delay detection may be done directly on the time delay or indirectlythrough detecting phase difference in the modulated signals.

The reference SPAD array 1316 senses the transmission of the transmittedoptical pulse signal 1302 generated by the light source 1300, andgenerates a transmission signal TR indicating detection of transmissionof the transmitted optical pulse signal. The reference SPAD array 1316receives an internal reflection 1318 from the lens 1304 of a portion ofthe transmitted optical pulse signal 1302 upon transmission of thetransmitted optical pulse signal from the light source 1300. Thereference SPAD array 1316 effectively receives the internal reflection1318 of the transmitted optical pulse signal 1302 at the same time thetransmitted optical pulse signal is transmitted. In response to thisreceived internal reflection 1318, the reference SPAD array 1316generates a corresponding SPAD event, and in response thereto generatesthe transmission signal TR indicating transmission of the transmittedoptical pulse signal 1302.

The delay detection and processing circuit 1314 includes suitablecircuitry, such as time-to-digital converters or time-to-analogconverters, to determine the time duration between the transmission ofthe transmitted optical pulse signal 1302 and receipt of the reflectedor return optical pulse signal 1306. The delay detection and processingcircuit 1314 then utilizes this determined time delay to determine thedistance D_(TOF) between the object 210 and the TOF proximity sensor120. The range estimation circuitry 1310 further includes a lasermodulation circuit 1320 that drives the light source 1300. The delaydetection and processing circuit 1314 generates a laser control signalLC that is applied to the laser modulation circuit 1320 to controlactivation of the laser 1300 and thereby control transmission of thetransmitted optical pulse signal 1302. The range estimation circuitry1310 also determines the signal amplitude SA based upon the SPAD eventsdetected by the return SPAD array 1312. The signal amplitude SA is basedon the number of photons of the return optical pulse signal 306 receivedby the return SPAD array 1312. The closer the object 210 is to the TOFproximity sensor 120, the greater the sensed signal amplitude SA, and,conversely, the farther away the object, the smaller the sensed signalamplitude.

FIG. 10 is a functional diagram of a single zone example of the returnSPAD array 1312 of FIG. 9. In this example, the return SPAD array 1312includes a SPAD array 1400 including a plurality of SPAD pixels SC, someof which are illustrated and labeled in the upper left portion of theSPAD array. Each of these SPAD pixels SC has an output, with two outputslabeled SPADOUT1, SPADOUT2 shown for two SPAD pixels by way of examplein the figure. The output of each SPAD pixel SC is coupled to acorresponding input of an OR tree circuit 1402. In operation, when anyof the SPAD pixels SC receives a photon from the reflected optical pulsesignal 1306, the SPAD pixel provides an active pulse on its output.Thus, for example, if the SPAD pixel SC having the output designatedSPADOUT2 in the figure receives a photon from the reflected opticalpulse signal 306, then that SPAD pixel will pulse the output SPADOUT2active. In response to the active pulse on the SPADOUT2, the OR treecircuit 1402 will provide an active SPAD event output signal SEO on itsoutput. Thus, whenever any of the SPAD pixels SC in the return SPADarray 1400 detects a photon, the OR tree circuit 1402 provides an activeSEO signal on its output. In the single zone example of FIG. 10, the TOFproximity sensor 120 may not include the lens 1309, and the return SPADarray 1312 corresponds to the return SPAD array 1400 and detects photonsfrom reflected optical pulse signals 1306 within the single detectionrange of the sensor.

FIG. 11 is a functional diagram of a multi-zone example of the returnSPAD array 1312 FIG. 9. In this example, the return SPAD array 1312includes a return SPAD array 1414 having four array zones ZONE11-ZONE14,each array zone including a plurality of SPAD pixels. Four zonesZONE11-ZONE14 are shown by way of example and the SPAD array 1414 mayinclude more or fewer zones. A zone in the SPAD array 1414 is a group orportion of the SPAD pixels SC contained in the entire SPAD array. TheSPAD pixels SC in each zone ZONE11-ZONE14 have their output coupled to acorresponding OR tree circuit 1406-1 to 1406-4. The SPAD pixels SC andoutputs of these pixels coupled to the corresponding OR tree circuit1406-1 to 1406-4 are not shown in FIG. 11 to simplify the figure.

In this example, each of zones ZONE11-ZONE14 of the return SPAD array1414 effectively has a smaller sub-detection range corresponding to aportion of the overall detection range of, e.g., the example sensor 120of FIG. 10. The return lens 1309 of FIG. 9 directs return optical pulsesignals 1306 from the corresponding spatial zones or sub-detectionranges within the overall detection range to corresponding zonesZONE11-ZONE14 of the return SPAD array 1414. In operation, when any ofthe SPAD pixels SC in a given zone ZONE11-ZONE14 receives a photon fromthe reflected optical pulse signal 1306, the SPAD pixel provides anactive pulse on its output that is supplied to the corresponding OR treecircuit 1406-1 to 1406-4. Thus, for example, when one of the SPAD pixelsSC in the zone ZONE11 detects a photon that SPAD pixel provides anactive pulse on its output and the OR tree circuit 1406-1, in turn,provides an active SPAD event output signal SEO1 on its output. In thisway, each of the zones ZONE11-ZONE14 operates independently to detectSPAD events (i.e., receive photons from reflected optical pulse signals306 in FIG. 9).

FIGS. 12 and 13 are graphs illustrating operation of the TOF proximitysensor 120 in detecting multiple objects. The graphs of FIGS. 12 and 13are signal diagrams showing a number of counts along a vertical axis andtime bins along a horizontal axis. The number of counts indicates anumber of SPAD events that have been detected in each bin, as will bedescribed in more detail below. These figures illustrate operation of ahistogram-based proximity technique implemented by the TOF proximitysensor 120. This histogram-based proximity technique allows the TOFproximity sensor 120 to sense or detect multiple objects within thedetection range of the TOF proximity sensor.

This histogram-based proximity technique is now described in more detailwith reference to FIGS. 9, 10, 11, 12 and 13. In this technique, morethan one SPAD event is detected in each cycle of operation, where thetransmitted optical pulse signal 1302 is transmitted in each cycle. SPADevents are detected by the return SPAD array 1312 (i.e., return SPADarray 1400 or 1414 of FIGS. 10, 11) and reference SPAD array 1316, wherea SPAD event is an output pulse provided by the return SPAD arrayindicating detection of a photon. Thus, an output pulse from the OR treecircuit 1402 of FIG. 10 or one of the OR tree circuits 1406-1 to 1406-4of FIG. 11. Each pixel in the SPAD arrays 1312 and 1316 will provide anoutput pulse or SPAD event when a photon is received in the form of thereturn optical pulse signal 1306 for target SPAD array 1312 and internalreflection 1318 of the transmitted optical pulse signal 1302 for thereference SPAD array 1316. By monitoring these SPAD events an arrivaltime of the optical signal 1306, 1318 that generated the pulse can bedetermined. Each detected SPAD event during each cycle is allocated to aparticular bin, where a bin is a time period in which the SPAD event wasdetected. Thus, each cycle is divided into a plurality of bins and aSPAD event detected or not for each bin during each cycle. Detected SPADevents are summed for each bin over multiple cycles to thereby form ahistogram in time as shown in FIG. 14 for the received or detected SPADevents. The delay detection and processing circuit 1314 of FIG. 9 orother control circuitry in the TOF proximity sensor 120 implements thishistogram-based technique in one example of the sensor.

FIGS. 12 and 13 illustrate this concept over a cycle. Multiple pixels ineach of the SPAD arrays 1312 and 1316 may detect SPAD events in eachbin, with the count of each bin indicating the number of such SPADevents detected in each bin over a cycle. FIG. 13 illustrates thisconcept for the internal reflection 1318 of the transmitted opticalpulse signal 1302 as detected by the reference SPAD array 1316. Thesensed counts (i.e., detected number of SPAD events) for each of thebins shows a peak 1500 at about bin 2, with this peak being indicativeof the transmitted optical pulse signal 1302 being transmitted. FIG. 12illustrates this concept for the reflected or return optical pulsesignal 1306, with there being two peaks 1502 and 1504 at approximatelybins 3 and 9. These two peaks 1502 and 1504 (i.e., detected number ofSPAD events) indicate the occurrence of a relatively large number ofSPAD events in the bins 3 and 9, which indicates reflected optical pulsesignals 1306 reflecting off a first object causing the peak at bin 3 andreflected optical pulse signals reflecting off a second object at agreater distance than the first object causing the peak at bin 9. Avalley 1506 formed by a lower number of counts between the two peaks1502 and 1504 indicates no additional detected objects between the firstand second objects. Thus, the TOF proximity sensor 120 is detecting twoobjects 210, within the detection range. The two peaks 1502 and 1504 inFIG. 12 are shifted to the right relative to the peak 1500 of FIG. 13due to the time-of-flight of the transmitted optical pulse signal 1302in propagating from the TOF proximity sensor 120 to the two objects 210within the detection range but at different distances from the TOFproximity sensor.

FIG. 14 illustrates a histogram generated by TOF proximity sensor 120over multiple cycles. The height of the rectangles for each of the binsalong the horizontal axis represents the count indicating the number ofSPAD events that have been detected for that particular bin overmultiple cycles of the TOF proximity sensor 120. As seen in thehistogram of FIG. 14, two peaks 1600 and 1602 are again present,corresponding to the two peaks 1602 and 1604 in the single cycleillustrated in FIG. 12. From the histogram of FIG. 14, the TOF proximitysensor 120 determines a distance to each of two objects 210 in thedetection range of the TOF proximity sensor 120. In addition, the TOFproximity sensor 120 also generates the signal amplitude SA for each ofthe objects 210 based upon these counts, namely the number of photons orSPAD events generated by the return SPAD array 1312 in response to thereturn optical pulse signal 306.

FIG. 15 is a diagram illustrating multiple spatial zones within thedetection range where the TOF proximity sensor 120 is a multi-zonesensor including the return SPAD array 1414 of FIG. 11. In such amulti-zone TOF proximity sensor 120 as functionally illustrated in FIG.15, the return lens 1309 (FIG. 9) is configured to route return opticalpulse signals 1306 from each of the object zones 1210 (11) to 1210(14)within the overall detection range 1210 of TOF proximity sensor 120 to acorresponding array zone ZONE11-ZONE14 of the return SPAD array 1414 ofFIG. 4B. This is represented in the figure through the pairs of lines1700 shown extending from the return SPAD array 1414 to each of theobject zones 1210(11)-1210(14).

Each of the array zones ZONE11-ZONE14 outputs respective SPAD eventoutput signals SE01-SE04 as previously described with reference to FIG.11, and the TOF proximity sensor 120 accordingly calculates fourdifferent imaging distances D_(TOF1)-D_(TOF4), one for each of theobject zones 1210(11)-1210(14). Thus, in this example the rangeestimation signal RE generated by the TOF proximity sensor 120 includesfour different values for the four different detected imaging distancesD_(TOF1)-D_(TOF4). Each of these detected imaging distancesD_(TOF1)-D_(TOF4) is shown as being part of the generated rangeestimation signal RE to have an example value “5”. This would indicateobjects in each of the object zones 1210(11)-1210(14) are the samedistance away, or indicate that the object is relatively flat within thedetection range of the TOF proximity sensor 120.

As the description illustrates, a single TOF proximity sensor 120 mayachieve the multi-zone distance detection and may be able to obtain the3D profile of a portion of an object 210 within the detection range 1210of the single TOF proximity sensor 120. Sensor controller 154 maycontrol the routing of the return optical pulse signals 1306 to adjustthe detection range 1210 (or sub-detection ranges) of a single TOFproximity sensor 120. Therefore, proximity sensing system 120 mayinclude multiple TOF proximity sensors 120 each having multi-zonedetection capacity and may include a single TOF proximity sensor havingmore powerful multi-zone detection capacity, which are all included inthe disclosure.

The various examples described above can be combined to provide furtherexamples. All of the U.S. patents, U.S. patent application publications,U.S. patent applications, foreign patents, foreign patent applicationsand non-patent publications referred to in this specification or listedin the Application Data Sheet are incorporated herein by reference, intheir entirety. Aspects of the examples can be modified, if necessary toemploy concepts of the various patents, applications and publications toprovide yet further examples.

These and other changes can be made to the examples in light of theabove-detailed description. In general, in the following claims, theterms used should not be construed to limit the claims to the specificexamples disclosed in the specification and the claims, but should beconstrued to include all possible examples along with the full scope ofequivalents to which such claims are entitled. Accordingly, the claimsare not limited by the disclosure.

What is claimed is:
 1. A robotic device, comprising: a base; a moveableportion coupled to the base; a time-of-flight proximity sensing systemcoupled to at least one of the base or the moveable portion, thetime-of-flight proximity sensing system being configured to detect adistance of an object to the time-of-flight proximity sensing system;and a control system configured to automatically control a reaction ofthe robotic device based on the detected distance of the object, whereinthe control system is configured to automatically: receive multipledetected distances of the object in a time sequence; determine amovement direction of the object based on the multiple detecteddistances; compare the movement direction of the object with a movementof the moveable portion; and control the movement of the moveableportion based on a result of the comparing.
 2. The robotic device ofclaim 1, wherein the time-of-flight proximity sensing system includes alight emitting element and a photonic diode-based light sensing element.3. The robotic device of claim 1, wherein the control system is furtherconfigured to determine a movement speed of the object based on themultiple detected distances.
 4. The robotic device of claim 1, whereinthe control system is further configured to determine a movement path ofthe object.
 5. The robotic device of claim 4, wherein the control systemcompares the determined movement path of the object with a movementtrajectory of the moveable portion in determining the reaction.
 6. Therobotic device of claim 1, wherein the time-of-flight proximity sensingsystem includes a first time-of-flight proximity sensing system coupledto the base and a second time-of-flight proximity sensing system coupledto the moveable portion, and wherein a priority between the firsttime-of-flight proximity sensing system and the second time-of-flightproximity sensing system in controlling the movement of the moveableportion is set based on the distance of the object.
 7. The roboticdevice of claim 1, wherein the time-of-flight proximity sensing systemis structured to detect a three-dimensional profile of the object. 8.The robotic device of claim 1, wherein the control system is furtherconfigured to adjust a detection range of the time-of-flight proximitysensing system.
 9. The robotic device of claim 8, wherein, thetime-of-flight proximity sensing system includes a first time-of-flightproximity sensor and a second time-of-flight proximity sensor, and thecontrol system controls at least one of the first time-of-flightproximity sensor or the second time-of-flight proximity sensor to moveits detection range away from one another based on a difference betweena first distance detected by the first time-of-flight proximity sensorand a second distance detected by the second time-of-flight proximitysensor.
 10. The robotic device of claim 8, wherein, the time-of-flightproximity sensing system includes a first time-of-flight proximitysensor and a second time-of-flight proximity sensor, and control systemcontrols at least one of the first time-of-flight proximity sensor andthe second time-of-flight proximity sensor to move its detection rangecloser to one another based on a difference between a first distancedetected by the first time-of-flight proximity sensor and a seconddistance detected by the second time-of-flight proximity sensor.
 11. Therobotic device of claim 1, wherein controlling the reaction of therobotic device includes controlling an operation speed of the moveableportion.
 12. The robotic device of claim 1, wherein controlling thereaction of the robotic device includes controlling a movement of thebase.
 13. The robotic device of claim 1, further comprising mapping theobject into a reaction zone of multiple reaction zones about themoveable portion based on the detected distance.
 14. The robotic deviceof claim 13, wherein controlling the reaction of the robotic device isbased on the reaction zone that the object is mapped into.
 15. Therobotic device of claim 1, wherein the control system is configured to:receive, from the time-of-flight proximity sensing system, a firstdistance of the object detected by a first time-of-flight proximitysensor of the time-of-flight proximity sensing system and a seconddistance of the object detected by a second time-of-flight proximitysensor of the time-of-flight proximity sensing system; compare the firstdistance and the second distance; and control a movement of a movableportion based on at least one of the first distance or the seconddistance and based on a result of the comparing.
 16. The robotic deviceof claim 15, wherein the control system is configured to: determine athree-dimensional profile of the object based on the first distance andthe second distance; and determine a classification of the object bycomparing the three-dimensional profile of the object with a parameterthree-dimensional profile.
 17. A robotic device having a proximitysensing system, comprising: a base; a moveable portion coupled to thebase; multiple proximity sensors, each proximity sensor being coupled toone of the base or the moveable portion and structured to detect adistance of an object to the proximity sensor; and a controllercommunicatively coupled to each of the multiple proximity sensors, thecontroller being configured to automatically control at least one of ascanning speed or a detection resolution of the multiple proximitysensors based on the distance of the object detected by a proximitysensor of the multiple proximity sensors and to automatically control areaction of the robotic device based on the detected distance of theobject, wherein the controller is configured to automatically: receivemultiple detected distances of the object in a time sequence; determinea movement direction of the object based on the multiple detecteddistances; compare the movement direction of the object with a movementof the moveable portion; and control the movement of the moveableportion based on a result of the comparing.
 18. The robotic device ofclaim 17, wherein the controller is configured to decrease the scanningspeed and increase the detection resolution of the multiple proximitysensors when the distance of the object is smaller than a firstthreshold, and is configured to increase the scanning speed and decreasethe detection resolution of the multiple proximity sensors when thedistance of the object is larger than a second threshold.